Techlead & Data engineer at Lectra
Data engineer at Lectra
TESTS !
it did not test the integration with the Kafka cluster
how to test several streams ?
Listen, let’s DEV do their job !
how to communicate with DEV/PO/QA ?
Integration tests with simple syntax
Integration tests with simple syntax
Integration with Kafka Streams
HTTP-based APIs
Peter Thomas - 2017
simple syntax (Gherkin)
Inner tool
console-script confluent
Cucumber Scala, using specific Gherkin DSL.
Feature: upper-case
Background:
Given input topic
| topic | alias | key_type | value_type |
| topic-simple-value | topic_in | string | string | (1)
And output topic
| topic | alias | key_type | value_type | readTimeoutInSecond |
| topic-simple-value | topic_string_out | string | string | 5 |
| topic-upper-case-string | topic_out | string | string | 5 | (2)
And var myKey = call function: uuid
Scenario: My first scenario
When records with key and value are sent (3)
| topic_alias | key | value |
| topic_in | ${myKey} | a |
| topic_in | ${myKey} | b |
| topic_in | ${myKey} | c |
Then expected records (4)
| topic_alias | key | value |
| topic_string_out | ${myKey} | input_1 |
| topic_string_out | ${myKey} | input_2 |
| topic_string_out | ${myKey} | input_3 |
| topic_out | ${myKey} | result_1 |
| topic_out | ${myKey} | result_2 |
| topic_out | ${myKey} | result_3 |
And assert input_1 $ == "a" (5)
And assert input_2 $ == "b"
And assert input_3 $ == "c"
And assert result_1 $ == "A"
And assert result_2 $ == "B"
And assert result_3 $ == "C"test infra
simple to use
tool to communicate
documentation
acceptance tests
docker build -t kapoeira:latest .
docker run --rm -ti \ -v <PATH_TO_YOUR_FEATURES_FOLDER>:/features \ -v /var/run/docker.sock:/var/run/docker.sock \ -e KAFKA_BOOTSTRAP_SERVER=<HOST:PORT[,HOST2:PORT2,HOST3:PORT3,...]> \ -e KAFKA_SCHEMA_REGISTRY_URL=<URL> \ -e KAFKA_USER=<XXX> \ -e KAFKA_PASSWORD=<****> \ -e JAAS_AUTHENT=<true (default) | false> \ -e LOGGING_LEVEL=<INFO (default) | ERROR | ...> \ -e THREADS=<8 (default) | ... > \ lectratech/kapoeira
TODO
docker compose build --no-cache docker compose up -d docker restart kapoeira
Feature: Burger π feature
Background:
Given input topic
| topic | alias | key_type | value_type |
| bread | bread-in | string | string |
| vegetable | vegetable-in | string | string |
| meat | meat-in | string | string |
And output topic
| topic | alias | key_type | value_type | readTimeoutInSecond |
| burger | burger-out | string | string | 5 |
And var uuid = call function: uuid
Scenario: Nominal
When records with key and value are sent
| topic_alias | key | value |
| bread-in | π€€_${uuid} | π |
| vegetable-in | π€€_${uuid} | π
|
| meat-in | π€€_${uuid} | π₯© |
Then expected records
| topic_alias | key | value |
| burger-out | π€€_${uuid} | order |
And assert order $ == "π"
Scenario: Not a burger
When records with key and value are sent
| topic_alias | key | value |
| bread-in | π€€_${uuid} | π |
| vegetable-in | π€€_${uuid} | π₯ |
| meat-in | π€€_${uuid} | π₯© |
Then expected records
| topic_alias | key | value |
| burger-out | π€€_${uuid} | order |
And assert order $ == "π + π₯ + π₯©"
Scenario Outline: Many customers
When records with key and value are sent
| topic_alias | key | value |
| bread-in | <user>_${uuid} | <bread> |
| vegetable-in | <user>_${uuid} | <vegetable> |
| meat-in | <user>_${uuid} | <meat> |
Then expected records
| topic_alias | key | value |
| burger-out | <user>_${uuid} | order |
And assert order $ == "<result>"
Examples:
| user | bread | vegetable | meat | result |
| π€€ | π | π
| π₯© | π |
| π | π | π
| π | π |
| π‘ | π | π
| π | π |Feature: Meal π feature
Background:
Given input topic
| topic | alias | key_type | value_type |
| bread | bread-in | string | string |
| vegetable | vegetable-in | string | string |
| meat | meat-in | string | string |
| side-dishes | side-dishes-in | string | string |
And output topic
| topic | alias | key_type | value_type | readTimeoutInSecond |
| meal | meal-out | string | string | 20 |
And var uuid = call function: uuid
Scenario: Left Join with Left first
When records with key and value are sent
| topic_alias | key | value |
| bread-in | π€€_${uuid} | π |
| vegetable-in | π€€_${uuid} | π
|
| meat-in | π€€_${uuid} | π₯© |
| side-dishes-in | π€€_${uuid} | π₯πΊ |
Then expected records
| topic_alias | key | value |
| meal-out | π€€_${uuid} | notif |
| meal-out | π€€_${uuid} | order |
And assert notif $ == "π"
And assert order $ == "π(π + ππΊ)"
Scenario: Left Join with Right first
When records with key and value are sent
| topic_alias | key | value |
| side-dishes-in | π€€_${uuid} | π₯π· |
| bread-in | π€€_${uuid} | π |
| vegetable-in | π€€_${uuid} | π
|
| meat-in | π€€_${uuid} | π₯© |
Then expected records
| topic_alias | key | value |
| meal-out | π€€_${uuid} | order |
And assert order $ == "π(π + ππ·)"